The Effects of Arts-Integrated Instruction on Students’ Memory for Science Content: 
Results from a Randomized Control Trial Study 

Mariale M. Hardiman, EdD 
Johns Hopkins University 

Ranjini Mahinda JohnBull, PhD 
Johns Hopkins University 

Deborah Carran, PhD 
Johns Hopkins University 

Paper presented at the annual meeting of the American Educational Research Association, 

April 2017 


1 



Abstract 


Strong correlational evidence suggests that involvement in the arts improves students’ academic 
outcomes and memory of learning events (e.g., Peppier et al., 2014; Robinson, 2013; Scripps & Paradis, 
2014). It is unclear, however, whether the improved outcomes are the result of general exposure to the 
arts, arts integrated into content instruction, the use of effective instructional practices, or a combination 
of these factors. Moreover, as a growing number of studies suggest that arts-integrated pedagogy 
enhances learning, few empirical studies have explicitly examined the direct effect of an arts-integrated 
curriculum on learning and specifically on students’ memory for non-arts academic content. Thus, this 
study sought to determine the effects of arts-integrated lessons on long-term memory for science 
content. We hypothesized that embedding arts-based activities into conventionally taught lessons would 
produce learning outcomes as good as or better than traditional instruction. This paper describes the 
results of a randomized control trial that measured retention of science content using arts-integrated 
science units and matched units employing convention science instruction. The study was conducted in 
16 fifth-grade classrooms in an urban mid-Atlantic school district. 
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Background 


Memories associated with being exposed to the arts are powerful—arts experiences are thought 
to elicit emotional cognition, employ creative thinking pathways, and recruit cognitive processes that 
inherently facilitate long-tenn recall. Strong correlational evidence suggests that involvement in the arts 
improves students’ academic outcomes and memory of learning events (e.g., Peppier et ah, 2014; 
Robinson, 2013; Scripps & Paradis, 2014). It is unclear, however, whether the improved outcomes are 
the result of general exposure to the arts, arts integrated into content instruction, the use of effective 
instructional practices, or a combination of these factors. Moreover, as a growing number of studies 
suggest that arts-integrated pedagogy enhances learning, few empirical studies have explicitly examined 
the direct effect of an arts-integrated curriculum on learning and specifically on students’ memory for 
non-arts academic content. Thus, this study sought to determine the effects of arts-integrated lessons on 
long-term memory for science content. We hypothesized that embedding arts-based activities into 
conventionally taught lessons would produce learning outcomes as good as or better than traditional 
instruction. This paper reviews the results of a randomized control trial that measured retention of 
science content using arts-integrated science units and matched units employing convention instruction. 
The study was conducted in 16 fifth-grade classrooms in an urban mid-Atlantic school district. 

Literature Review/Conceptual Framework 

In the following sections, we discuss how arts integration is described and outline the conceptual 
framework for our investigation of the effects of arts-integrated pedagogical methods on students’ 
retention of academic content. We argue that, while arts-involvement is correlated with better student 
learning outcomes, the majority of the extant research involves quasi-experimental or correlational 
research designs, which leaves room to question causality of increased performance outcomes. Finally, 
in this section we outline our initial theory of change and provide evidence supporting the hypothesized 
effects of arts-integrated instruction on memory for non-arts academic content. 
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Arts education and arts integration. The variety of arts instructional methods and arts 
integration within education has been approached in several ways: 

(a) art forms such as dance, drama, visual arts, or music are taught in discrete classes focused on 
students’ acquisition of arts standards; (b) arts-based activities are employed as a means to teach 
other academic areas or concepts in non-arts content (e.g. shaping the body like a particular letter 
to enhance emerging literacy through dance or movement); or (c) the arts are used to reinforce 
academic concepts and make the content more engaging (e.g., using warm and cool colors when 
creating maps in science class). (Peppier et ah, 2014, p. 366) 

Arts integration has also been described as a means to promote the effective transfer of knowledge and 
skills from arts to non-arts domains and to help students draw connections among different disciplines 
(Bumaford, Brown, Doherty, & McLaughlin, 2007). Specifically, Burnaford et al. (2007) describe arts 
integration as: learning academic content with and through the arts to enhance learning outcomes; arts 
activities as a curricular connections process; and arts-infused learning as a way to foster collaborative 
engagement within a learning activity. A more recent definition delineates the process as a co-equal 
cognitive integration of the arts where “the arts are integrated with other aspects of the curriculum and 
students are required to use higher-order thinking skills and aesthetic qualities to gain further 
understanding of a particular academic concept” (Robinson, 2013, p. 192). For our study, we define arts 
integration as a pedagogical method for teaching non-arts academic content in which both non-arts and 
arts-based standards are addressed. 

Arts integration and student learning outcomes. A plethora of research on the effects of arts 
education and arts-integrated pedagogy on student outcomes has been conducted over the past thirty 
years, but the majority of this literature largely examines the correlations between arts study (described 
as some type of involvement in the arts), and academic outcomes (Butzlaff, 2000; Podlozny, 2000; 
Vaughn & Winner, 2000; Winner & Cooper, 2000). Within the smaller body of arts integration 
literature, evidence indicates that arts-integrated instruction correlates with higher levels of reading and 
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mathematics achievement (Robinson, 2013). Most of these studies involve experimental or quasi- 
experimental designs in which the various factors that contribute to achievement may not be controlled. 
Still, it is important to examine findings involving arts integration at the whole school level that show 
promise for improving student learning outcomes. 

For example, in Peppier et al.’s (2014) three-year quasi-experimental study of matched treatment 
(N= 3) and control (N= 3) schools, researchers investigated the effects of arts-integrated English 
language arts (ELA) instruction on standardized ELA assessments. This study used the Learning and 
Achieving Through the Arts model, which provides three strands to arts-integrated programming: (a) 
instruction to promote art skills and language development for students, (b) in-depth professional 
development and coaching for non-arts teachers, and (c) supplementary activities to encourage whole 
school adoption of arts integration to buttress creative experiences in all classrooms. While baseline 
scores on the standardized test indicated that control schools included more students initially at the 
proficient level, treatment schools significantly increased the percentages of students in proficient 
categories (increases of 10-13%) compared to no increase in the students in the proficient ELA test 
categories (-3% to 0% change) in control schools. Further, the average increase in ELA proficiency 
among English-language learners from baseline was 15% across all three years of the study in the arts- 
integrated schools. While these results support the effectiveness of arts integration, the findings are 
limited because the data analyzed for the study were school-based scores and not individual student test 
data, where, if available, the intricacies of the intervention might be parsed further or where the impact 
of the intervention might be examined more closely (Pepper et ah, 2014). 

Similarly, Scripps and Paradis (2014) implemented a quasi-experimental study design examining 
the effects of arts-focused schools versus academic-focused schools to detennine the causal li nk s 
between arts integration experiences, teacher professional development concerning arts integration, and 
student academic and arts outcomes. Students in the arts-focused programs outperformed students in 
paired academic-focused schools on arts-based assessments and standardized tests; academic-focused 
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schools’ pre- and post- academic scores remained fixed. In addition, low performing students in the arts- 
integrated schools largely reached the achievement of their average and high performing peers; a 
statistically significant difference in scores no longer existed. These findings suggest that applications of 
arts-based interventions may have a more powerful impact on academic outcomes for students from 
low-perfonning groups than students from proficient and advanced groups. 

In a recent meta-analysis of arts integration studies, Robinson (2013) evaluated 453 studies of 
arts integration to examine the effects of arts integration on low SES students’ outcomes. Forty-four 
studies met the criteria for examining arts integration as a method to promote knowledge and skill in 
content areas and, in addition, influenced domains of cognition and motivation. Results indicated that 
arts integration, especially the use of multiple arts forms, had positive effects on reading achievement 
for populations of high-need students. There was also positive correlational evidence of the benefits of 
multi-arts integration on mathematics achievement, creativity/critical thinking, self-efficacy, motivation, 
cooperation, and student engagement. Along with the previously reviewed studies, this meta-analysis 
lends support for the possibility that arts integration improves student learning and broader cognitive 
domains such as creative thinking and problem-solving. 

Arts integration and memory. While these studies suggest correlational evidence that 
involvement in the arts improves students’ academic outcomes (e.g., Peppier et al., 2014; Robinson, 
2013; Scripps & Paradis, 2014), it is not clear whether the improved outcomes are the result of arts 
exposure, integrating the arts into content instruction, strong teacher beliefs and practices, or a 
combination of these factors. Moreover, few empirical studies have explicitly examined the effect of arts 
integration methods on students’ memory for non-arts academic content through pre-, post-, and delayed 
content-based assessments specifically designed for the study. Recent theoretical papers and studies on 
arts integration support the theory of enhanced memory when students leam non-arts content through 
arts-integrated pedagogy. 
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For example, Hardiman (2003, 2010, 2012) and Rinne, Gregory, Yarmolinskaya, and Hardiman 
(2011) describe the potential cognitive benefits of the arts on long-tenn learning. Rinne et al. (2011) 
delineate a theory of change related to arts integration’s effect on students’ retention of non-arts 
academic content. Arts integration pedagogical methods use multiple modes of learning that allow 
students to engage in learning activities unlike traditional curricular methods. To that end, the authors 
discuss the cognitive science underpinnings of the benefits of (a) rehearsal (e.g., the repetition of content 
through song or rap), (b) elaboration (e.g., drawing an example of known content), (c) generation (e.g., 
dramatizing an interaction between two famous scientists), (d) enactment (e.g., demonstrating states of 
matter with one’s body), (e) oral production (e.g. singing the movements of the Earth), (f) effort after 
meaning (e.g., deciphering artistic renderings of the solar system), (g) emotional arousal (e.g., imagining 
the wonder of a first look through a telescope for early astronomers), and (h) pictorial representation 
(e.g., examining different artistic renderings of plant cells). Taken together, the authors argue that these 
“memory effects” are naturally recruited through the arts. Thus, this conceptual framework describes 
how arts integration methods use research-based, memory-enhancing activities to potentially improve 
memory for non-arts content (see Hardiman et al., 2014). 

Based upon the aforementioned framework, Hardiman et al. (2014) conducted a study of arts- 
integrated methods on memory for science content in one mid-Atlantic school within four fifth-grade 
classrooms using a randomized control experimental design. Curriculum specialists developed science 
units in the topics of Astronomy and Environmental Science using arts-integrated activities for the 
treatment units and conventional instruction for the control units. Unit pairs were matched so that they 
provided the same science content, the same dosage of each content component, and the same mode of 
delivery to assure active learning experiences in both conditions. Pre, post, and delayed post-test data 
were collected using curriculum-based assessments designed to examine the retention of content taught 
in the units. Results indicated that arts-integrated instruction produced a statistically significant 
difference overall for memory of science content (p = .012). The driver of this outcome was a strong 
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statistically significant effect for students reading at the lowest levels of reading achievement (p = .009). 
Using scores from the annual state assessment to determine reading performance, data showed that 
students scoring at the basic level benefitted the most from art-integrated instruction compared to peers 
performing at the proficient and advanced reading levels. The limitations of this study included using 
curricular materials that were not scripted, which led to variability in implementation fidelity. These 
findings corroborate Scripp and Paradis’ (2014) results suggesting that students who perfonn at lower 
levels of reading achievement may benefit more from arts-integrated methods than their higher¬ 
performing peers. 

Initial Theory of Change 

Our theory of change is based on aforementioned literature that describes how arts-integrated 
instruction may increase students’ memory for academic content. In the following sections, we describe 
our study examining arts integration and memory, provide initial evidence of the effects of arts- 
integrated instruction on memory, and offer observations that the same activities may simultaneously 
facilitate broader domains of learning such as creative problem-solving. 

Rationale/Purpose 

What remains missing from the quasi-experimental studies described above (i.e., Peppier et ah, 
2014; Scripps & Paradis, 2014) and Robinson’s (2013) meta-analysis is an examination of arts- 
integrated pedagogy using a research design that can draw causal relationships between arts-integrated 
instruction and content performance outcomes. The study of Hardiman et al. (2014) begins to identify 
causal evidence suggesting that arts-integrated methods impact long-tenn memory of non-arts content. 

In our current study, we aimed to build from these findings through a randomized control trial that used 
a larger sample size and controlled for fidelity of implementation by scripting all content into teacher 
guide books. 

We used the same method for matching treatment and control units described in the Hardiman et 

al. (2014) study, revised the two science unit pairs employed in that study, and developed two additional 
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science unit pairs in the topics of Life Science and Chemistry. All four unit pairs included detailed 
directions for implementing science and arts-integrated activities, companion student workbooks, 
PowerPoint presentations, and science and art lesson materials. Curriculum-based assessments were 
used to measure the long-term retention of academic content through pre, post, and delayed testing. 

Methods 

Participants 

Student Participants. A total of 350 students from 16 fifth grade classrooms across six schools 
were eligible to participate. Two schools had four fifth grade classrooms, and four schools had two fifth 
grade classrooms each. 

Classroom pairs were matched within sites to create a balanced design controlling for time of 
day, within teacher effects, and order of the treatment. All teachers delivered both arts-integrated and 
conventional instruction. Each participant was exposed to both arts-integrated and conventional science 
instruction. Each science unit was taught over the course of 3-4 weeks during the fall 2013 semester, and 
this length of time is defined as a “session.” The study was implemented for two sessions to ensure that 
participants experienced both control and treatment conditions. Students were randomly assigned to 
treatment and control groups for the first unit of study in the first session of this project. In the second 
session of the study, the participants stayed in their randomized groups and entered the opposite 
treatment condition for a second science unit. The participants received the order of treatment based on 
their random assignment. They received one treatment condition in the first session in one subject, and 
then received the other treatment condition in the second session with a different science unit. For 
example, a student who was randomly assigned to conventional Astronomy in session 1 received arts- 
integrated Life Science in session 2. 

Teacher Participants. All teachers were rated as high quality, and each teacher had served at 
least two years at their schools prior to this study. 


Setting 
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This study took place in a school district located in the mid-Atlantic region of the United States. 
The district enrolls approximately 85,000 students in grades preK through grade 12, 83% African- 
American, 84% low income based on Free and Reduced Meal Services, includes 186 schools, and has a 
budget of $ 1.34 billion. Schools were selected to participate based on the number of fifth grade 
classrooms. In order to be included in the study, each school needed at least two fifth grade classrooms 
or four fifth grade classrooms to allow for matching. Schools were also selected based upon previous 
relationships with the principal investigator. 

Design 

This study was designed as a randomized control trial with randomly assigned equivalent control 
groups with condition reversal for the second unit of curriculum. We randomized at the student level by 
using a random number generator and placed students in one of two treatment conditions, either arts- 
integrated science or conventional science, for the first session of the study. Then, student IEP 
designations were examined to ensure that the students with IEPs were evenly distributed across the 
classes. In the second session of the study, students stayed in their randomized class assignments and 
experienced a second science unit in the alternate treatment condition. For example, if a student was 
assigned to arts-integrated Life Science in the first session of the study, then in the second session, the 
student would take conventional Astronomy. There were three testing phases of pretest, posttest, and 
delay test. 

Independent Variable 

Arts-integrated curriculum involved teacher delivery of curricular content and student 
demonstration of knowledge of content through visual and performing arts. The research team 
developed four treatment units that use arts-integrated instructional strategies in four different science 
topics and four control units matched in content that use conventional instruction. In order to control for 
potential confounding factors, we closely matched the treatment and control units in terms of content, 
dosage (the amount of time for content delivery for each activity across treatment and control 
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conditions), order of content presentation, and type of instructional activity (e.g., group, individual, 
paired). As the curriculum writers designed the units, they were careful to develop activities in both 
conditions that adhered to those matching requirements. While some of the activities were the same in 
both conditions, the control condition mostly involved conventional teacher-directed instruction through 
presentations, videos, PowerPoint presentations, and textual readings. Students demonstrated acquisition 
of content through oral and written activities. In the treatment condition, teacher presentations often used 
arts forms to convey the concept and students displayed understanding by engaging in a variety of visual 
and perfonning arts. The following examples demonstrate the differences between conditions: In the 
control condition, students displayed knowledge through designing a chart or presenting the information 
orally, whereas in the arts-integrated treatment condition they displayed knowledge through a variety of 
arts-based activities such as dance, tableau, singing, or drawing. In the control condition, students 
expanded on their understanding of vocabulary by writing a sentence using the target word, whereas in 
treatment they demonstrated their understanding of the vocabulary by taking visual notes, which entailed 
drawing sketches and writing notes. To reinforce content, students in the control condition engaged in 
choral reading of specific passages; in the treatment condition they sang a song or a chanted a rap. 

The topics of the four unit pairs for this study are Astronomy, Life Science, Chemistry, and 

Environmental Science. Astronomy and Environmental Science were revised units, and Chemistry and 

Life Science were new units developed for this study. The pairs of science units included one revised 

unit and one new unit; Astronomy was paired with Life Science, and Environmental Science was paired 

with Chemistry. Each unit covers 15 days of instruction; each of the 15 lessons include standards based 

on the Next Generation Science Standards as well as Common Core State Standards. The unit fonnat 

follows the 5E Learning Cycle model ( Engage, Explore, Explain, Elaborate, and Evaluate) (Bybee et 

al., 1989). The units include overarching goals using a graphic organizer/concept map for each unit, 

instructional objectives, vocabulary, and activities for each of the components of the 5E Learning Cycle 

model. We used the software Adobe InDesign for the lesson templates, which allowed for color-coding 
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general instructions to teachers, scripted teacher presentation language, and desired student responses. 
We scripted the lessons to promote consistency in content and instructional delivery. Based on our 
experiences with the preliminary study, we believed that scripting the lessons was an important 
component of the unit design, as some elementary teachers expressed a lack of confidence in their 
knowledge of science content and in effectively executing science activities. Teachers were asked not to 
read from the script but instead to carefully review the lessons so that they could deliver them in a 
natural way. 

For each unit, teachers received lesson plan guide books, student workbooks, and all materials 
that were needed to deliver the science and the arts-based activities. They also were provided with lap 
top computers if they did not have access to technology in their classrooms and all supplementary 
electronic media, including PowerPoint presentations, videos, and music. 

Dependent Variables 

Curriculum-based assessments. The research team identified key content for items for 
curriculum-based assessments in the four content areas and developed pre, post, and delayed tests. The 
tests consisted of 30 items in a multiple-choice format with four possible responses. We created three 
versions of each assessment for pre/post/delayed testing by changing question wording or modifying the 
order of answer choices. The measures were deliberately designed to be difficult to avoid ceiling effects 
and make retention challenging. 

After completion of the study, posttest content assessment responses were examined to identify 
item difficulty. Any item that received less than 10% correct responses was dropped from the test pool 
across testing times. No items were dropped from the Astronomy assessment, seven items were deleted 
from the Life Sciences assessment, one item was deleted from the Chemistry assessment, and four items 
were dropped from the Environmental assessment. All students’ scores were standardized for analyses. 

Fidelity of implementation checklists. Fidelity of implementation checklists were designed 
based on recommended best practices as outlined in O’Donnell (2008) and Nelson et al. (2012). The 
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four areas included: a) Exposure - The degree to which students are “exposed to” or receiving the 
treatment or control; b) Adherence - The degree to which the teacher is implementing the lessons as 
written; c) Participant Responsiveness - The degree to which students are participating in the lesson 
activities; and d) Quality of Delivery - The degree to which the teacher demonstrates that he/she is 
prepared to teach the lesson ( e.g. presentation of lesson is not read directly from script but taught in a 
natural way and all materials are prepared and ready when needed). Observers rated the four areas above 
on the following scale: 0 indicating no evidence; 1 indicating weak evidence; 2 indicating partial 
evidence; and 3 indicating strong evidence. A section titled “notes” was included for comments and 
evidence collected for each of the four areas. A final notes section asked observers to record any issues 
that may have arisen during the lesson delivery such as disruptions to instruction due to occurrences 
such as fire drills, public address announcements, visitors to the classroom, etc. They also noted any 
departures from delivering the lessons as written. For example, they noted content that may have been 
skipped or implemented in shorter or longer time periods than allocated. Observers were present in the 
classrooms from 40% to 60% of instructional time in both the treatment and control conditions. Fidelity 
of implementation data indicated that teachers implemented lessons to fidelity at least 90% of the time 
during the observed lessons. 

Materials 

The research team reviewed each lesson in the eight units (60 lesson plans) to determine the 
materials that would be required to teach the science and arts-integrated activities in both the 
conventional and arts-integrated conditions. Materials lists were compiled and items purchased for 32 
groups of students (approximately 360 students - 16 arts-integrated science kits and 16 conventional 
science kits) for science and arts-integrated activities. The research team recruited volunteers from area 
high schools and universities to create, assemble, and deliver materials kits to each study site school. 

The research team reviewed materials with study site teachers and observers during the professional 
development sessions. 
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Procedure 


Professional development for study site teachers and fidelity of implementation observers 
occurred several weeks before the start of the study in early fall of 2013. The professional development 
was designed to assure that study site teachers had appropriate command of science content for the units 
they would teach and with the arts activities that were integrated into the units. The teachers received ten 
hours of formal training, which consisted of reviewing the activities that were designed for each day of 
instruction for each condition. Additionally, the training included simulation activities for targeted arts- 
integrated activities and science experiments. The study site teachers received all materials at the 
training sessions and learned how the materials were to be used with specific activities. In addition to the 
formal professional development, members of the research team provided additional one-on-one 
coaching to individual study site teachers throughout the study as needed. 

The first units of the study were implemented in 16 classrooms across six schools in early fall of 
2013. We labeled this first time period as the first quarter and the second set of unit implementation the 
second quarter. For each content area in each quarter, half of the units were taught in the treatment 
condition and half in the control condition. For example, the Chemistry units were taught by four 
teachers; two who taught in the arts-integrated condition and two who taught in the control condition in 
the first quarter. In the second quarter, the teachers taught the reversed condition to a different group of 
randomized students. This was done for all curricular units. 

Analysis 

To determine the effects of arts-integrated instruction on long-tenn retention of content, 
descriptive and central tendency statistics were employed examining the differences between arts- 
integrated and conventional instructional methods. In the first analyses, we sought to detennine whether 
arts-integrated instruction affected retention in all students across all units combined. Next, we 
examined the effects of arts-integrated instruction at the different reading levels. Then, we examined the 
treatment effects at the level of the unit pairs (Astronomy-Life Science and Chemistry-Environmental 
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Science) alone and with the reading levels. Finally, we investigated the effects of both the instructional 
methods and the order of instructional methods on the percent of retained content at the level of the unit 
pairs. Long-term retention of science content was operationalized by detennining the percentage of 
retained content. The following formula was utilized: ((Retained Content (T3) - Prior 
Knowledge(Tl))/(Initially Learned Content(T2) - Prior knowledge(Tl))* 100. In other words, we 
suggest that this formula represents the percentage of science content that was retained over time while 
controlling for students’ prior knowledge. 

Results 

Arts-integrated Instruction versus Conventional Instruction: All Science Units 

Across all science units. A one-way analysis of variance was employed to examine the effects 
of different instructional methods on long-term retention of science content. The instructional methods 
factor included two different conditions: arts-integrated science instruction and conventional science 
instruction. Homogeneity of variance was upheld by Levene’s test which indicated that the groups were 
not significantly different and were derived from the same population; F( 1,508) = 0.146, p > 0.05. 

Results of the analysis indicated a non-statistically significant difference between percent of 
retained content in the arts-integrated instruction condition versus the conventional instruction condition 
(MAI = 64.22%, MC = 67.99%), F(l, 508) = 0.128 ,p = 0.721. 

Across all science units and reading levels. In order to examine the effects of instructional 
methods on long-tenn retention, we added reading ability levels as a second factor to the ANOVA 
model. Using predetermined benchmarks from the state end of year assessment, three reading levels 
were identified as basic, proficient, and advanced. A 2 x 3 factorial analysis of variance was employed 
to inspect the effects of the instructional methods at the different reading levels (basic, proficient, 
advanced) on percent of retained content. Homogeneity of variance was not upheld by Levene’s test 
which indicated that the six groups may have been different and that the error variance was was not 
evenly distributed across the groups; F(5,475) = 5.2 ,p < 0.05/6. 


15 



Results of the factorial model revealed non-statistically significant main effects between percent 


of retained content in the arts-integrated instruction condition versus the conventional instruction 
condition, F(l, 475) = 2.101 ,/? = 0.148. A non-statistically significant difference was observed between 
the reading levels, F(2, 475) = 0.343,/? = 0.710. An interaction effect was observed between 
instructional method and reading levels, F(2, 475) = 3.570,/? = 0.029. This interaction effect indicates 
that for students reading at basic levels of proficiency, learning through arts-integrated methods yielded 
greater memory of the science content than learning science through conventional methods. It also 
indicates that proficient and advanced readers remembered approximately the same amount of science 
content from both the arts-integrated lessons and the conventional science lessons. 


Table 1 Mean Percentages of Retained Content in the Reading Levels and Instructional Methods 


Reading 

Level 

Instructional 

Method 

Mean 

Std. 

Deviation 

N 

Basic 

Conventional 

32.3% 

172.7 

30 


Arts-integrated 

105.8% 

150.1 

30 

Proficient 

Conventional 

72.75% 

125.5 

123 


Arts-integrated 

69.0% 

124.1 

118 

Advanced 

Conventional 

69.7% 

63.8 

89 


Arts-integrated 

53.7% 

80.5 

91 


Results of the factorial model revealed non-statistically significant main effects between percent 
of retained content in the arts-integrated instruction condition versus the conventional instruction 
condition, F(l, 475) = 2.101,/? = 0.148. A non-statistically significant difference was observed between 
the reading levels, F(2, 475) = 0.343,/? = 0.710. An interaction effect was observed between 
instructional method and reading levels, F(2, 475) = 3.570,/? = 0.029. Basic readers remembered 
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significantly more science content learned through the arts at the delayed post-test than basic readers 
who learned science through conventional methods. 

Arts-integrated Instruction versus Conventional Instruction: Unit Pairs 

Unit pairs. A 2 x2 factorial analysis of variance was used to examine the effects of instructional 
method (arts-integrated and conventional) and unit-pairs (Astronomy-Life Science and Chemistry- 
Environmental Science) on mean percent of retained content. Homogeneity of variance was upheld by 
Levene’s test which indicated that the four groups were not significantly different; F(3, 506) = 3.195, /? 

> 0.05/4. 

Findings of the factorial model revealed a non-statistically significant difference between percent 
of retained content in the arts-integrated instruction condition versus the conventional instruction 
condition (MAI = 64.22%, MC = 67.99%), F(1 ,506) = .094 ,p = 0.759. For unit pairs, a non-statistically 
significant difference was observed on percent of content retained; (MAS-FS = 59.61%, MCH-ES = 
73.78%), F(l, 506) = 1.801,/? = .180. No interaction effect was observed between instructional method 
and unit pairs, F(l, 506) = 0.275,/? = 0.600. 

Arts-integrated Instruction versus Conventional Instruction, Unit Pairs, and Timing of 
Instructional Method 

A 2 x 2 x 2 factorial analysis of variance was used to examine the effects of instructional method 
(arts-integrated and conventional), unit-pairs (Astronomy-Life Science and Chemistry-Environmental 
Science), and timing of the instructional method (Session 1 or Session 2) on mean percent of retained 
content. Homogeneity of variance was upheld by Fevene’s test which indicated that the four groups 
were not significantly different; F(7, 502) = 2.02,/? > 0.05/4. 

Findings of a 2 x 2 x 2 factorial ANOVA revealed a non-significant main effect between percent 
of retained content in the arts-integrated instruction condition versus the conventional instruction 
condition for the Chemistry-Environmental Science unit pair, F(l, 502) = 1.516,/? = 0.219. A non- 
statistically significant main effect was found in the percent of retained content in the arts-integrated 
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instruction condition versus the conventional instruction condition, F(1 502) = .003,/? = 0.954. A non- 
statistically significant main effect of percent of retained content was found for timing of instructional 
method, F( 1,502) = 0.907 ,p = 0.341. One interaction effect was significant at/; < .10. A statistically 
significant interaction between Condition by Timing, F (1,502) = 3.091,/? = 0.079 was observed. This 
means that students who took arts-integrated science in the first session remembered even more science 
in the second session when they learned science through conventional instructional methods. 


Table 2 Mean Percentages of Retained Content between Instructional Methods and Different Sessions 


Timing 

Instructional 

Method 

Mean 

Std. 

Deviation 

N 

Session 1 

Conventional 

70.9% 

105.4 

142 


Arts-integrated 

51.3% 

122.7 

135 

Session 2 

Conventional 

64.3% 

128.4 

114 


Arts-integrated 

78.9% 

119.7 

119 


Conclusions 

Overall, the hypotheses were confirmed. Using arts-integrated instruction to teach science 
content was as effective as or better than conventional science instruction in increasing long-term 
memory for students’ science content knowledge. A combined analysis of all units in treatment and 
control conditions showed an advantage for arts-integrated instruction; statistically significant 
differences were found only within an individual unit pair. This was not unexpected, as findings from 
previous studies (e.g., Peppier et ah, 2014; Scripps & Paradis, 2014) suggest that only one year of 
exposure to arts-integration would not likely produce statistically detectable differences in academic 
outcomes. 

It is important to note that the results from this study mirrored the findings of Hardiman et al. 
(2014) in that the groups of students reading at basic levels as measured by standardized reading 
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assessments benefited the most from arts-integrated instruction compared to proficient and advanced 
readers. We found significant differences in retention of science content in the arts-integrated condition 
compared to the control condition for this group of learners. 

We also found interesting trends of treatment and order effect. While all groups performed better 
on the second set of instructional units, students who were in the control condition in the first session 
performed at higher levels in the arts-integrated condition during the second session. Students who had 
arts-integrated instruction in the first session, however, performed just as well when in the control 
condition in the second session. 

This leads us to wonder whether there are transfer effects in which students may be applying 
creative problem solving skills (Dunbar, 2008; Gregory et ah, 2013; Hardiman, 2012; Limb & Braun, 
2008; Sawyer, 2006) and the acquisition of art skills and competencies to better understand and 
remember enduring ideas or big ideas (Bumaford et al., 2007; NAEA, 2002; NCCAS, 2014; Robinson, 
2013). Further, we hypothesize that students who learned science through the arts in the first session 
may have transferred some creative insights and arts competencies to their learning strategies within the 
conventional science lessons in the second session. In addition, from our conversations and qualitative 
responses from teachers following the study, feedback suggests that teachers may also have felt more 
confident and efficacious with teaching the units in the second session of the study (Ashton & Webb, 
1986; Gibson & Dembo, 1984; Tschannen-Moran & Hoy, 2007). Because this was not one of our 
original research questions for this particular study, we did not analyze these results with this potential 
variable in consideration, rather, we used their feedback to inform the revisions of the curricular units. 
Finding patterns of greater confidence for the second session units within the feedback from the teachers 
was an unexpected result which we plan to parse further in upcoming studies. 

Implications for Research and Practice 
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This pilot study supported findings from a preliminary study (Hardiman et al., 2014) with 
evidence suggesting that arts-integrated instruction is as effective as or better than conventional 
instruction for long-term memory of science content. Consistent with earlier findings, students 
performing at the lowest levels of reading achievement benefit the most from this pedagogical method. 

Unexpectedly, we also found evidence that arts-integrated instruction may produce a transfer 
effect after students have been exposed to arts integration first, a potential treatment-by-order effect. 
Future research exploring additional variables related to the nature of the interaction between timing of 
the intervention and content retention might help explain this possible transfer effect. 

Additionally, we pose that future studies should further explore the effect of arts-integrated 
instruction on memory for content for students at the lower levels of reading achievement. Perhaps given 
alternate and engaging ways to leam and demonstrate mastery of content through arts-integrated 
instruction, students who are experiencing challenges in reading acquisition may improve perfonnance 
and engagement in learning. Exploring the efficacy of using the arts as a pedagogical tool could begin to 
shed light on how to address the performance gap that continues to challenge educators looking for 
viable and scalable solutions to differentiated instructional approaches. Moreover, we propose future 
studies to examine the extent to which potential variables of student self-efficacy and teacher efficacy 
mediate memory for content, creative problem solving skills, arts skills, and conceptual competencies. 


This study was conducted with support from the U.S. Department of Education, Institute of Education 
Sciences, No. R305A120451 
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See Hardiman, M. (2015). The Effects of Arts-Integration on Retention of Content and Student 
Engagement. Grant No.R305A120451, Washington DC: Institute of Education Sciences, Johns Hopkins 
University School of Education. 
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